The present invention relates to a way to control the operation of a video system by enabling it to recognize one of several known visual patterns in the field of view of a video camera. Control of a video system, such as a video game or virtual reality system, through the visual recognition and tracking of a passive control device, or prop, is known in the art. Such a method of control allows the user considerable freedom of motion, and allows a number of different control devices to be provided at low cost, since the control device need not contain any electronics, but needs only have visual characteristics suitable for furnishing control inputs.